1. Field of the Invention
This invention relates to an image processing method wherein an optimum image processing is performed automatically on photograph image data such as digital photograph image, and the image is evaluated, to an image processing apparatus, a medium on which an image processing control program is recorded, an image evaluation device, an image evaluation method, and a medium on which an image evaluation program is recorded.
2. Description of the Prior Art
Various kinds of image processing may be performed on digital image data, i.e., in which processing: contrast may be increased; color may be corrected; or lightness may be corrected. This image processing can usually be performed with a microcomputer. An operator confirms the image on a monitor, the necessary image processing is selected, and image processing parameters are determined.
In recent years various types of image processing techniques have been proposed, and are now having considerable impact. However, a human operator is still required when it is a question of which technique to apply, and to what extent it should be used. This is because it was otherwise impossible to determine which digital image data had to be subjected to image processing. For example, in the case of image processing to correct the lightness of an image, the screen is made lighter if it is dark on the whole, and is made darker if it is too light.
Now, consider the case of a photographic image of a person filmed at night, where the background is near to pitch-darkness but the person in the picture has been well photographed. If this photograph is automatically corrected, it is attempted to make the image brighter due to the fact that the background is pitch black, so the final image appears as if the photo was taken in the daytime.
In this case, if a human operator is involved, he pays attention only to the person in the picture. If the image of the person is dark, it would be made a little brighter, conversely darkening would be selected if the effect of flash, etc., was too bright.
Hence, there was a problem in the prior art in that a human operator had to participate to determine the important part (referred to hereafter as the xe2x80x9cobjectxe2x80x9d) of a photographic image.
However, even when the importance of the image is evaluated by some technique, the determination process is performed in picture element units, and varying the importance in real time causes an increase in computation.
It is therefore an object of this invention, which was conceived in view of the aforesaid problems, to provide an image processing method which permits an important part of a photographic image such as a digital photograph image to be detected, and an optimum image processing to be automatically selected, to provide an image processing apparatus, and to provide a medium on which an image processing control program is recorded.
In order to achieve the aforesaid object, this invention is an image processing apparatus into which photographic image data comprising dot matrix picture elements is input, and which performs predetermined image processing on the input data. The apparatus comprises an image processing acquiring unit which acquires the aforesaid photographic image data, an image processing indicator specifying unit which performs predetermined summation processing on picture elements and specifies an image processing indicator based on the acquired image data, and a processing unit which determines image processing contents based on the specified image processing indicator, wherein the aforesaid image processing indicator specifying unit comprises an object determining unit which determines picture elements having a large image variation amount to be those of the object, and the aforesaid processing unit determines image processing contents based on image data for picture elements determined to be those of the object and performs image processing on the determined contents.
Herein, it is assumed that in the case of an image of a photographic image of a person, the person is usually photographed in the center of the field. Therefore, the person is brought into focus to give a sharp image. When the image is sharp, the outline part becomes clear, and the amount of image variation becomes large. As a result, there is an extremely high possibility that there will be no error if it is assumed that picture elements with a large image variation amount are those of the original object which has been brought into focus.
In the invention thus comprised, in the image processing indicator specifying unit based on photographic image data comprising dot matrix picture elements acquired in the image data acquiring unit, predetermined summation processing is performed on picture elements. This summation processing may take various forms, and the image processing indicator is basically specified based on the summation result. In the processing unit, the image processing contents are determined based on the specified indicator, and the determined image processing is performed.
This means that useful information about the corresponding image can be obtained by performing summation processing on dot matrix photographic image data, and image processing is performed on the data. In this way, the image processing indicator is specified by actual photographic image data, so optimum image processing can be performed even without human intervention.
The photographic image data comprises dot matrix picture elements and image processing is performed in picture element units, but first, in the object determining unit, picture elements having a large image variation are determined to be those of the object. Image processing contents are then determined in the processing unit based on image data for picture elements determined to be those of the object, and image processing is performed based on the determined contents.
Therefore, according to this invention, the determination of the object, which in the past required human participation, can be automated by determining the object as picture elements with a large image variation. The invention therefore provides an image processing method whereby optimum image processing can be performed by suitably modifying the image processing contents according to the object.
This method for specifying an image processing indicator from actual photographic data may of course be applied not only to a real device but also to a system on the method. In such a sense, this invention is also an image processing method wherein photographic image data comprising dot matrix picture elements is input, and predetermined image processing is performed, this image processing method comprising an image data acquiring step for acquiring the aforesaid photographic image data, an image processing indicator specifying step for performing a predetermined summation processing on picture elements based on this acquired image data, and specifying an image processing indicator, and a processing step for determining image processing contents based on the specified indicator, and performing image processing, wherein the aforesaid image processing indicator specifying step comprises an object determining step wherein picture elements having a large image variation amount are determined to be those of the object, and wherein in the aforesaid processing step, image processing contents are determined based on image data for picture elements determined to be those of the object, and image processing is performed based on the determined image processing contents. In this case, the apparatus offers all the attendant benefits and advantages of the invention.
This apparatus for determining an object and performing image processing method may be implemented by a stand-alone apparatus as mentioned above, or may be incorporated in another instrument which comprises such an apparatus. In other words, the scope of this invention covers various forms of implementation. It may also be implemented by hardware or software, and can be modified as necessary.
When the apparatus for implementing the concept of this invention is implemented by software, the invention applies equally to media on which this software is recorded and which can be used in exactly the same way. In this sense, this invention is also a recording medium whereon an image processing control program is recorded for inputting photographic image data comprising dot matrix picture elements by a computer, and which performs predetermined image processing on the input data. The control program comprises an image processing indicator specifying step for acquiring the aforesaid photographic image data, an image processing indicator specifying step for performing predetermined summation processing on picture elements and specifying an image processing indicator, and a processing step for determining image processing contents based on the specified image processing indicator, wherein the aforesaid image processing indicator specifying step comprises an object determining step which determines picture elements having a large image variation amount to be those of the object, and in the aforesaid processing step, image processing contents are determined based on image data for picture elements determined to be those of the object, and image processing is performed on the determined contents. In this case, the recording medium offers all the attendant benefits and advantages of the invention.
The recording medium may of course be a magnetic recording medium, an optomagnetic recording medium, or any kind of recording medium which may be developed in the future. It will of course be understood that the medium may be a first copy or second copy, and that a telecommunication line may also be used to supply the program. In this case, there is no difference regarding the application of the invention. There is also no difference if the program is written on a semiconductor chip.
There is no difference as regards the concept of the invention even if one part is software, and one part is implemented with hardware, or when it is in such a form that one part is stored on a recording medium which can be read when necessary.
Photographic image data means image data obtained when it is attempted to take a photograph of a real object. Image processing tries to correct images by comparing the images with the real objects from which they were acquired. The invention therefore applies not only to natural objects but also to manmade ones. More specifically, this includes image data read by a scanner, or image data captured by a digital camera.
Various techniques may be employed to determine the variation of picture elements in the object determining step. A further object of this invention is to give a specific example of this.
In the image processing method provided by this invention, in the aforesaid object determining step, the amount of variation of picture elements is determined based on a difference between adjacent picture elements.
Hence according to this invention, in the object determining unit where an image variation amount is determined, the determination is performed based on a difference of image data between adjacent picture elements. When there is a fixed interval between picture elements as in the case of a dot matrix, the difference of data between adjacent picture elements is directly proportional to a first order differential. This difference can be taken as the variation amount of the image. In this case the difference is regarded as the magnitude of a vector, and the vector may also be constructed by considering adjacent directions.
According to this invention, only the difference of image data between adjacent picture elements is found. Computation is therefore easy, and the processing amount for object determination can be reduced.
The determination of an object is of course not limited to this technique, and it is a further object of this invention to provide other examples.
According to the image processing apparatus provided by this invention, in the aforesaid object determining unit, the criterion for determining whether or not there is a large image variation amount changes according to the position of the image.
In the case of a photograph for example, a person is often photographed in the center. In this case it may be said that in order to determine image processing content, the picture elements to be determined as the object should be selected from the central part of the field. However, it may be said that whether or not there is a large variation in the image depends on a difference from a comparison value, and there is no reason why such a value always has to be constant.
Therefore, to determine whether or not there is a large image amount according to this invention, in the object determining unit, this criterion is altered depending on the position of the image, the criterion for each position being compared with the image variation amount of each picture element.
Hence, according to this invention, the assessment of image variation changes depending on the position of the image, and a highly flexible determination which considers image composition is thus possible.
The criterion can be altered in various ways. As one example, a certain trend may be ascertained, or alternatively, a trend which causes a change may be read from the image.
A further object of this invention is to provide an example of the former.
In the image processing apparatus provided by this invention, in the object determining unit, the aforesaid criterion is set lower for the central part than for the edges of the image.
By setting the criterion lower for the center than for the edges, it is easier to determine the center part of the image as the object even if the variation amount at the center and at the edges is approximately the same. Therefore if there is an image of a person in the central part, the picture elements of this person will be determined as the object more frequently.
According to this invention, a determination can be made which gives more weight to the center area of a photograph, and a large amount of image data can be effectively processed.
A further object of this invention is to provide an example of the latter when the criterion is varied.
According to the image processing apparatus provided by this invention, in the aforesaid object determining unit, the above criterion is based on the distribution of the aforesaid image variation amount at different points on the image.
Hence according to this invention, in the object determining unit, the distribution of image variation is found in each part of the image, and the aforesaid criterion is determined after finding this distribution. Subsequently, a comparison is made with this criterion to determine whether or not the picture elements are those of the object.
According to this invention, as the object is determined taking account of the distribution of image variation for picture elements, the image data can be treated flexibly.
When the criterion is determined based on distribution, it may be considered that there is a high possibility of finding the object in a part where there are many picture elements with a large variation amount, and the criterion maybe set low.
Alternatively, basic setting patterns may first be prepared according to a variation distribution pattern, and a basic setting pattern may then be chosen based on the detected distribution pattern.
At the same time, assuming that the image processing indicator specifying unit comprises such an object determining unit, the image processing contents may be determined based on image data which is determined to be that of the object, and image processing may then be performed on the determined contents, there being no limitation on the specific processing method employed. For example, a luminance distribution of picture elements determined to be those of the object is found, and if the luminance distribution range is enlarged in a predetermined proportion when the luminance distribution is narrow, image processing to increase contrast is performed. If the luminance distribution of the object seems dark on the whole, a correction may be made to make it lighter. The color distribution of picture elements determined to be those of the object is found, and it is determined whether or not the grey balance is off. If it seems to be off, tone curves are used to modify the grey balance.
Hence, the importance of the image has an effect even if the image data is summed in order to specify the image processing indicator. However, even if the importance of the image is determined by some technique, the work is carried out in picture element units, so varying the importance of an image in real time implies an increase of computational amount.
A further object of this invention is to consider the importance of photographic image data such as digital photograph images in relatively simple terms, and perform optimum image processing automatically.
The image processing apparatus provided by this invention is an apparatus for inputting photographic image data comprising dot matrix picture elements, and performing predetermined image processing. This image processing apparatus comprises an image data acquiring unit for acquiring the aforesaid photographic image data, an image processing indicator specifying unit which performs a predetermined summation processing on picture elements based on this acquired image data and specifies an image processing indicator, and a processing unit which determines image processing contents based on the specified indicator and performs image processing. The aforesaid image processing indicator specifying unit comprises a feature amount uniform sampling unit which determines an image processing intensity by uniformly sampling a feature amount over a whole screen, and a feature amount weighting reevaluation unit which reevaluates the feature amount sampled in the feature amount sampling unit with a predetermined weighting. In the aforesaid image processing unit, the image processing intensity is determined based on the reevaluated feature amount, and image processing is performed with the determined intensity.
According to the invention having the above construction, photographic image data comprises dot matrix picture elements, and in the feature amount uniform sampling unit, the feature amounts of picture elements are uniformly sampled over the whole screen. In the feature amount weighting reevaluation unit, the feature amounts that are sampled in this feature amount uniform sampling unit are reevaluated with a predetermined weighting. Then, in the processing unit, the image processing intensity is determined based on the feature amounts that have been reevaluated in this way, and image processing is performed.
In other words, as the sampling is uniform over the whole screen and a predetermined weighting is applied after sampling, the feature amounts obtained as a result are different from what is obtained by uniform sampling over the whole screen.
According to this invention the sampling in the sampling stage is uniform over the whole screen, so the computational amount is not too high. At the same time, by applying a predetermined weighting after sampling, irrelevant evaluation is not made as it would be if the picture elements were merely sampled uniformly without weighting. The invention therefore provides an image processing apparatus in which optimum image processing can be performed automatically.
It will be understood that the technique of performing a uniform sampling in the sampling stage and applying a predetermined weighting thereafter, may be applied not only to a real device but also to a system in both of which cases it has all the attendant benefits and advantages of this invention. As a specific example of the concept of this invention, when the image processing apparatus is implemented in terms of software, there naturally exist recording media on which the software is recorded which can be used to perform the function of the invention.
The feature amount uniform sampling unit uniformly samples feature amounts over the whole screen, for determining the image processing intensity. For this purpose, all picture elements over the whole screen can be sampled, but it is not necessary to sample all of the picture elements if the sampling is uniform.
A further object of this invention is to provide an example of the latter case.
According to the image processing apparatus of this invention, in the aforesaid feature amount uniform sampling unit, the aforesaid feature amounts are sampled for selected picture elements after uniformly thinning out the picture elements according to predetermined criteria.
According to this invention, by thinning out the picture elements according to predetermined criteria, the number of picture elements to be processed is reduced, and the aforesaid feature amounts are sampled from the remaining elements.
Herein, the term xe2x80x9cuniform thinningxe2x80x9d comprises the case where picture elements are selected at a fixed interval, and the case where they are selected at random.
According to this invention, as the picture elements are thinned out when the feature amounts are uniformly sampled, the processing amount is reduced.
The sampled feature amounts are reevaluated by a predetermined weighting in the feature amount weighting reevaluation unit. The sampled feature amounts are in picture element units, but the weighting can be applied either to picture element units or to suitable aggregates of picture elements.
A further object of this invention is to provide an example of the latter case.
According to the image processing apparatus of this invention, in the aforesaid feature amount uniform sampling unit, feature amounts are sampled in area units that are divided according to predetermined criteria, and in the aforesaid feature amount weighting reevaluation unit, a weighting is set for each area and the feature amounts are then reevaluated.
The invention as formulated hereabove assumes weightings in area units of the image that are divided according to predetermined criteria. In the feature amount uniform sampling unit, feature amounts are sampled in these area units, while in the aforesaid feature amount weighting reevaluation unit, the feature amounts are reevaluated with weightings set for each area.
The division of these areas may always be constant, or it may be made to vary for each image. In this latter case the division method may be changed according to the contents of the image.
According to this invention, as the weighting is made to vary for each area, the computation is relatively simple.
Any type of weighting technique can be employed provided that reevaluation is performed without merely performing uniform sampling.
A further object of this invention is to provide an example of this.
According to the image processing apparatus of this invention, in the aforesaid feature amount weighting reevaluation unit, the aforesaid weighting is made to vary by a correspondence relation determined by the position of picture elements in the image.
In the case of a photograph, the person is usually in the center. Therefore, by weighting the central part of the image more heavily after uniformly sampling feature amounts from the whole image, the feature amounts sampled from picture elements relating to the person are evaluated to be larger.
When for example according to the invention thus comprised, the weighting of the central part of the image is heavier and the weighting of the surroundings is lighter, in the feature amount weighting reevaluation unit, the position of picture elements in the image is determined, and a reevaluation is made using a weighting which varies according to this position.
Hence according to this invention, as the weighting is determined according to the position of picture elements, the computation is relatively simple.
The weighting technique is of course not limited to this method, and a further object of this invention is to provide other examples.
According to the image processing apparatus provided by this invention, in the aforesaid feature weighting reevaluation unit, the image variation amount is found, and a heavier weighting is given to parts where the image variation amount is larger.
In the invention thus comprised, the image variation amount is found before performing reevaluation in the feature amount weighting reevaluation unit. The image variation amount is also known as image sharpness, and as the outline is sharper the better the focus, the variation is large where the image is in focus. On a photograph, the part which is in focus is the subject, and the part which is not in focus is considered to be the background. Therefore, places where there is a large image variation are considered to correspond to the subject. In the feature amount weighting reevaluation unit, the same result as sampling a large number of feature amounts is obtained by applying heavy weighting to parts where there is a large image variation.
According to this invention, as the weighting is varied depending on image sharpness, different targets can be precisely identified and feature amounts can be sampled for different images.
As another example of a weighting technique, in the weighting reevaluation unit of the image processing apparatus of the invention, the chromaticity of picture elements is found, a number of picture elements is found for which the chromaticity lies within the chromaticity range of the target for which it is desired to sample a feature amount, and heavier weighting is applied to parts where there are many of these picture elements.
Hence according to this invention, in the feature amount weighting reevaluation unit, the chromaticity of picture elements is found. In image processing, an object can sometimes be specified by a specific chromaticity. For example, there is no reason why a person could not be identified by looking for skin color, but it is difficult to specify skin color as color data also contain luminance elements. Chromaticity on the other hand represents an absolute proportion of a color stimulation value, and it is not controlled by luminance. Therefore an image of a person could be determined if the chromaticity was within a specified range that can be taken as indicative of skin color. This reasoning may of course also be applied to the green of the trees or the blue of the sky.
As specified objects can be sorted by chromaticity according to this invention, different targets may be precisely sampled depending on their images and feature amounts sampled.
Therefore in a feature amount weighting reevaluation unit, when the chromaticity found for picture elements is within a chromaticity range for a target from which it is intended to sample feature amounts, plural picture elements are counted. When the number of picture elements is large, this part of the image is determined to be the target, heavy weighting is applied, and a large feature amount is sampled from the target.
This weighting technique is not necessarily the only alternative, and it is a further object of this invention to provide a suitable example of overlapping methods.
According to the image processing apparatus of this invention, in the aforesaid feature amount weighting reevaluation unit, temporary weightings are applied based on a plurality of factors, and these factors are then added according to their degree of importance to give final weighting coefficients.
According to the invention as thus comprised, in the feature amount weighting reevaluation unit, temporary weighting coefficients are found separately based on a plurality of factors, and the weightings are added according to their degree of importance so as to reevaluate the sampled feature amounts as final weighting coefficients. Therefore, it may occur that even when a large weighting is assigned by one weighting method in the evaluation stage, if the method does not have a large importance, the final weighting which is assigned is not large. Moreover, it may occur that even if there is a large difference between weighting methods, image parts which are evaluated to have an average or higher weighting also have a large final weighting.
According to this invention, plural weighting techniques are suitably combined so that a suitable feature amount evaluation can be performed.
Therefore if the image processing indicator specifying unit itself comprises plural forms, it is not necessary to perform image processing with only one of these forms.
However even if there are some cases where it is desirable to perform image processing using the feature amount of the object, there are other cases where it is desirable to perform image processing using an average feature amount for the whole photographic image. For example, when a photograph is taken of a person, the person may not always be the lightest (highlighted) part of the picture. Therefore if attention is paid only to the person in the picture and contrast is increased, the highlighted part of the background will be too white. In this case, a better result would be obtained by paying attention to the whole photographic image.
Hence when image processing is performed, it is still necessary to select an optimum feature amount.
It is a further object of this invention to automatically select an optimum feature amount according to image processing technique.
In the image processing apparatus according to this invention, photographic image data comprising dot matrix picture elements is input, and predetermined image processing is performed. This image processing apparatus comprises an image data acquiring unit for acquiring the aforesaid photographic image data, an image processing indicator specifying unit for performing a predetermined summation processing on picture elements based on this acquired image data, and specifying an image processing indicator, and a processing unit for determining image processing contents based on the specified indicator, and performing image processing. The aforesaid image processing indicator specifying unit comprises an evaluation unit for obtaining a feature amount by inputting the aforesaid photographic data, summing the image data for all picture elements, and obtaining a feature amount according to plural predetermined evaluation criteria. In the aforesaid processing unit, the image data can be converted by plural techniques, and the feature amounts obtained in the aforesaid evaluation unit used according to the particular technique.
According to this invention, image data from a photographic image comprising dot matrix picture elements is input in this manner, and a feature amount is obtained according to plural evaluation criteria by summing image data for picture elements in the evaluation unit. In the processing unit, in converting the image data by plural techniques, the feature amounts obtained in the evaluation unit according to each technique are then used to convert the data depending on the technique.
Specifically, although there are cases where the image data is best converted using a feature amount centered on the object such as in light/dark correction, there are other cases where image data is better converted using a feature amount centered on the whole image such as when contrast is increased, and the image data conversion may be performed by suitably selecting these plural feature amounts.
According to this invention, when feature amounts are obtained according to plural evaluation criteria and image processing is performed by plural methods, the feature amounts used depend on the method, so image processing may be performed based on an optimum evaluation criterion.
When image data is converted, the feature amounts should be such that they can be used to identify the features of the image, and there is no need to specify the type of image. For example this also includes indicators such as luminance histograms which identify whether the image is to be considered as light or dark, there being no need to obtain the identification result that the image is light or dark. Apart from lightness, the indicator may of course specify whether or not the image is sharp, or it may be an indicator to identify vividness.
There is also no particular limitation on the way in which the evaluation unit and processing unit are applied. For example, assuming that plural image processing methods are used, plural feature amounts may be obtained and stored according to plural evaluation criteria, and the image data converted by suitably selecting the feature amount in the processing unit as necessary. As another example, image data for picture elements may be summed by a predetermined criterion in the evaluation unit so as to obtain a suitable feature amount on each occasion that image processing is performed in the aforesaid processing unit.
It will of course be understood that these plural image processing methods based on feature amounts obtained by plural different evaluation criteria, may also be applied not only to an actual device but also to a system both of which are then a valid form of the invention. When the image processing methods are implemented by software as specific examples of the concept of the invention, there naturally exist media on which the software is recorded which then offer all the attendant advantages thereof.
The evaluation criterion used to obtain feature amounts in the aforesaid evaluation unit will depend on the image processing that is to be performed, and while there are some cases where it is desirable to concentrate on the object for image processing, there are some cases where it is not as described above.
It is a further object of this invention to provide an example of the former case.
In the image processing apparatus according to this invention, the aforesaid evaluation unit comprises an evaluation unit wherein an object in a photographic image is sampled, and image data for picture elements of this object is summed to obtain a feature amount, and in the aforesaid processing unit, in one processing method, the feature amount obtained from object picture elements is used when the feature amount for the central part of the image data is used.
According to the invention thus comprised, when image data is converted based on the feature amount of the central part of the image data in the aforesaid processing unit, the object in the photographic image is sampled in the aforesaid evaluation unit, and the feature amount is obtained by summing image data for object picture elements according to predetermined criteria.
Herein, the central part of the image data has the following meaning. For example, when it is determined whether a given photograph is light or dark, it is easily appreciated that the determination can conveniently be based on the intermediate density of the image. This intermediate density may also be referred to as a median in a luminance distribution, i.e. the center of the luminance distribution, and in this sense it is referred to as the central part of the image data. Then, if there is an object in the photographic image, it may be said that there is a definite necessity to perform light/dark correction in line with the lightness of this object.
This invention is suitable for the case when image processing is performed based on the feature amount of the central part of the image data.
Any of the aforesaid techniques may be applied as the basic technique for sampling the object. As an example, in the aforesaid evaluation unit, picture elements for which there is a large variation of image data between adjacent picture elements are sampled as the object. When picture elements are aligned at a fixed interval apart as in the case of a dot matrix image, the difference of image data between adjacent picture elements is proportional to a first order differential. This difference may be determined as the image variation amount. In this case, the difference may be regarded as the magnitude of a vector, and the vectors constructed taking account of adjacent directions. If this is done it is sufficient to determine the difference of image data for adjacent picture elements, computing is easy, and the processing for determining the object is reduced.
As another example, in the aforesaid evaluation unit, picture elements for which the chromaticity is within a predetermined range may be sampled as the object. In this case, in the aforesaid evaluation unit, the chromaticity of picture elements is found. The chromaticity represents an absolute proportion of a color stimulation value, and it is not affected by lightness. Therefore the object in the image can be separated by possible range of chromaticity. For example, there is the chromaticity range for skin color, or the chromaticity range for the green of the trees. As this can be said for chromaticity, in the aforesaid evaluation unit, picture elements for which the chromaticity lies within a predetermined range are sampled as the object. In this way, an object can be determined by its chromaticity, and the object may be sampled without depending on the lightness or darkness of the object.
On the other hand as an example of image processing not concerned only with the object, the aforesaid evaluation unit comprises an evaluation criterion wherein picture elements of the aforesaid image data are uniformly sampled and summed so as to obtain a feature amount, and in the aforesaid processing unit, in one processing method, the feature amount obtained by the aforesaid uniform sampling is used when an average feature amount of the photographic image is used. In this case, when image data is converted based on the average feature amount in the aforesaid processing unit, the feature amount is obtained by uniformly sampling picture elements of the image data according to predetermined evaluation criteria. Of course, the summation may be performed on all picture elements of the photographic image, but it may be said that is no advantage as the processing amount increases. Hence, it is convenient to perform image processing based on the average feature amount of the photographic image, e.g. saturation correction.
As another example of image processing which is not concerned only with the object, the aforesaid evaluation unit comprises an evaluation criterion wherein picture elements of the aforesaid image data are uniformly sampled and summed to obtain a feature amount, and in the aforesaid processing unit, in one image processing method, the feature amount obtained by uniform sampling is used when the edges of a feature amount distribution of the photographic image are used. In this case, in the aforesaid processing unit, it is assumed that the ends of the feature amount distribution obtained in the aforesaid evaluation unit are used. For example, to increase the contrast, image processing is performed to find the luminance distribution, and the edges of this luminance distribution are widened, but if the luminance distribution of the object were used in this case, other highlighted parts appear white. Therefore in this case, in the aforesaid evaluation unit, picture elements of image data are uniformly sampled according to predetermined criteria, and summed to obtain the feature amount. This is suitable for image processing using the edges of a feature amount distribution in an actual photographic image, e.g. for increasing contrast.
In the above, a continuous sequence of processes is performed comprising predetermined analysis of the image and image processing by specifying image processing indicators, but the analysis result itself is also useful.
It is a further object of this invention to provide an image evaluation device wherein it is easier to use an image evaluation result which is an analysis result done.
In the image evaluation device offered by this invention, photographic image data comprising dot matrix picture elements is input, the image data for all picture elements is summed according to predetermined criteria, and is the image evaluation device which is based summation result, and evaluate image, and the image is evaluated based on the summation results. There are plural evaluation criteria for the aforesaid summation results, and the evaluation results are combined with a predetermined weighting based on these evaluation criteria.
According to the invention as thus comprised, the evaluation method assumes that photographic image data comprising dot matrix picture elements is input, the image data is summed for picture elements, and the image is evaluated based on the summation results. Herein, there are plural evaluation criteria for these summation results, and the evaluation results are combined with a predetermined weighting based on the evaluation criteria.
In other words, although some evaluation criteria are suitable for evaluating images where a sharp image is the object such as in the case of portrait, other criteria are suitable for evaluating images where the background is the important object. A general evaluation may be made by suitably combining plural evaluation criteria in parallel and varying the weightings.
As described hereabove, as this invention gives a general evaluation by varying the weightings of plural evaluation criteria, it provides an image evaluating device which can be flexibly adapted to image feature determination.
Naturally, the concept of this invention for image evaluation by the above techniques comprises many forms. Specifically, it comprises hardware and software, various modifications being possible as may be convenient. When the concept of the invention is implemented by image processing software, there naturally exist recording media on which the software is recorded which can be used to perform the function of the invention. Moreover, these image evaluating techniques may be applied to an image evaluating device and its system running on software media.
Various techniques may be employed to achieve the same object in applying plural evaluation criteria to summation results. For example, all picture elements may be summed by weighting with different evaluation criteria, but it will be appreciated a large amount of processing is involved when the summation is applied to all picture elements. Hence, as described above, the picture elements are first sampled based on plural evaluation criteria, summed, and the summation results combined with a predetermined weighting. In this case, the image data are sampled prior to summation, plural criteria are used by varying the criteria applied to the sampling, and the weighting of the summation results is then adjusted before combination. An evaluation can therefore be made with different weightings on the results based on plural evaluation criteria. Due to this sampling of image data, plural evaluation criteria may be employed according to the sampling method.
As one evaluation criterion, the data may of course be sampled uniformly and summed. In this case, the image data is uniformly thinned and the whole image is considered, which makes this a suitable criterion for determining scenic photographs, etc. In this way an optimum criterion can be used while reducing the processing amount.
As an example of a criterion which can be applied whether or not sampling is used, is the evaluation of picture elements which have a large variation relative to adjacent picture elements with a heavier weighting. In this case, the image variation of picture elements from adjacent elements is detected, and clear image parts with a large variation are given a heavier weighting in the summation. If this is done, as image parts with a large variation are often parts of the photograph which are clearly in focus, an image evaluation which weights important parts of the image can be performed.
This criterion places more emphasis on the sharp parts of the image, and it is therefore naturally suitable for the determination of human images. Herein, image parts with a large variation may be evaluated either by introducing weighting as the picture elements are summed, or by summing only picture elements with a large variation. The weighting used in the evaluation is not necessarily fixed, but may also be allowed to vary according to the criterion. In this case, by varying the weighting for different evaluation criteria, an overall evaluation result for the image can be deduced. Moreover various approaches are possible, e.g. weightings may be varied individually so as to generate plural combinations which are then selected. In this way, by modifying the weighting for plural criteria, a more flexible evaluation can be made.
Instead of an operator varying the weighting, this can be done based on the image data itself. As an example of this, the weighting of the evaluation results may be varied based on the criteria. In this case, results are obtained according to various criteria, and the weighting is modified in view of the suitability of the criteria in the light of the results. As the results are used to vary the weighting, the work involved in the evaluation is less.
Various techniques may also be used to modify the weighting of the criteria using the results. For example, if it is determined whether or not the image data for picture elements should be sampled according to one criterion, the number of these picture elements may be taken as a criterion and the weighting increased when the number of picture elements is large.